Conversation
Add `lock_holder_liveness` to the file-lock owner and `turn_lane_liveness` on top of it. Both classify a lane's last executing Turn from its holder record alone: `released` on a clean exit, `dead` when the record names this machine and the pid is gone, `foreign_host` when the pid cannot be checked here, `unreadable` when a lock file carries no parseable record, and `live` only when a same-host pid is still alive. The probe never touches the kernel lock. A probe that acquired it for an instant would refuse a real `run-once --execute` racing that instant with `turn_lane_in_flight` for nothing; the new test drives the real fence wrapper concurrently with a continuous probe and proves the Turn is admitted exactly once. The holder host label is now single-sourced so the writer and the readers cannot disagree on what "this machine" is. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: song <22676124+songoow@users.noreply.github.com>
Delegated members had no stopped observation: prepared, running and turn_returned could only end in accepted or rejected. Add "stopped" as a terminal observation reachable from the three open states and keep the inventory check driven by the same transition table. Add the exported collaboration.delegation.stop decision: a stop request settles only with an acknowledgement from a process that held the operation lock plus a free operation lock and a free Turn lane lock. Free locks with no acknowledgement are "unknown", never a fabricated settlement, and a grace timeout alone changes nothing while a lock is still held. Register the RPC handler next to the existing delegation observation handlers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: song <22676124+songoow@users.noreply.github.com>
…eceipts
Delegated members could not be stopped: the worker held the operation lock
for the whole run and rewrote the execution record from memory, so nothing
written into that record could reach it or survive it, and a killed run left
its Turn and hard lease unaccounted for.
Add a stop receipt beside the execution record (executions/<h>/<op>.stop.json)
written only under the existing .dispatch lock, never into the record. Its
phases are requested -> acknowledged -> settled with the terminals unknown and
noop, and it records the requester, the lock-holding worker (pid, process
group, host), the acknowledgement (pid, observed status, Turn key), the lease
release outcome and the settlement facts (operation lock, Turn lane lock,
Turn journal status).
Delegations.stop: a terminal record returns an identical noop receipt on every
call. Otherwise the request is written; when no worker holds the operation the
requester takes the lock, marks the record stopped, releases the hard lease
and settles. A same-host holder is SIGTERMed by process group (its run-once
child and host bridge follow) and SIGKILLed only if it still holds the
operation after the grace; another host's holder is never signalled and finds
the request itself. Settlement is the typed collaboration.delegation.stop
decision over lock facts: elapsed time is never a receipt.
Worker: a SIGTERM handler raises DelegationStopRequested once; checkpoints
before running, before each run-once and before completing the Todo raise on
a stop file; the record is written only through a fenced write that re-reads
the stop file under .dispatch and raises DelegationFenced for a stop this
process did not acknowledge, so a late-returning or other-host worker records
no Turn result and completes no Todo. Acknowledgement marks the record
stopped, releases the hard lease and leaves the in_progress Turn journal for
inspection. execute records the worker's pid/pgid/host at entry and
acknowledges from under the lock when a stop already exists.
resume refuses stopped work ("start a new operation id"), wait returns on
stopped, and read exposes the stop phase. file_lock gains local_lock_host and
read_lock_holder so the stop names the holder exactly as lock records do.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: song <22676124+songoow@users.noreply.github.com>
Add `loopx delegation stop --operation-id ID --execute` next to start/resume/adopt (--execute is required for the same reason) and the `stop_delegation(operation_id)` MCP tool, which runs the blocking stop off the event loop like wait_delegation. Both return the same receipt and never resume or rerun work. The inventory reader skips `<op>.stop.json` sidecars, which sit beside execution records but are not records, and the delegation context and subagent context projections count `stopped` receipts instead of dropping them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: song <22676124+songoow@users.noreply.github.com>
Semantics-first coverage for stopping delegated members: - stop while a detached worker runs a sleeping fixture host: the worker acknowledges SIGTERM under its own lock, the receipt settles only with a free operation lock and a lockable Turn lane, the record bytes stay frozen afterwards, the Todo stays open, the host and worker processes are gone, the Turn journal stays in_progress, and resume is refused without spawning; - stop after accepted: noop, identical on repeat, no stop file and artifacts unchanged; stop without a holder is acknowledged by the requester; - a worker SIGKILLed before acknowledging settles as unknown, not stopped, and resume stays refused; - a fenced write after another process's acknowledged stop raises and writes nothing, while an unacknowledged stop is taken from under the lock at entry; - CLI stop requires --execute and repeats its receipt; inventory pages past stop sidecars and reads a stopped record as stopped; - TS: stopped is terminal and reachable only from open observations, and the stop decision settles only on acknowledgement plus free locks. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: song <22676124+songoow@users.noreply.github.com>
Describe `delegation stop --execute` / `stop_delegation` in both reference documents, in English and Chinese: where the request lives, how a same-host worker is signalled and acknowledges, why another host's worker is left to find the request, what settled, unknown and noop prove, and that stopped work needs a new operation id while its Turn journal and open Todo remain for the coordinator to inspect. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: song <22676124+songoow@users.noreply.github.com>
… lock The stop settlement probed the member's Turn lane by acquiring its lock for an instant, which could refuse a legitimate Turn of the same member racing that instant with turn_lane_in_flight. It also carried its own host label helper next to the file-lock owner's. Settlement now reads the lane's last holder record through turn_lane_liveness: released, dead or absent frees the lane; a live same-host holder frees it only when it sits outside the recorded worker's process group; another host's holder, an unattributable holder or an unreadable record keep the stop open. The operation lock is read the same way, so a stop never refuses a legitimate resume or status read. The local host helper is deleted in favour of lock_holder_host_label. A regression test races a real lane acquisition against settlement and proves the Turn is admitted. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: song <22676124+songoow@users.noreply.github.com>
Regenerated with scripts/generate_project_registry_io_manifest.py after rebasing onto main. Site ids and classifications are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: song <22676124+songoow@users.noreply.github.com>
d5aafc8 to
5e7b7bf
Compare
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
精确 head：5e7b7bfd8fb425312d25a771911984040ac9ee03。按 LoopX PR review capability 检查整份差异，未把相邻 PR 的发现移植到本 PR。
动机
Roadmap R2 要求 stop/cancel/restart 保留工作并 fence 旧执行者。已有 delegated member 无可用停止入口，杀进程也不能证明结果、Turn 和租约已妥善收尾。本 PR 增加 CLI/MCP 停止与持久 ACK，是合理的有界交付；但用户需要的是“停止已完成”的可信结果，不能在返回 settled 后原执行宿主仍可继续工作。
改动思路
复用既有 binding、GoalRef、operation 单飞锁、dispatch 锁及 Turn lane owner。stop sidecar 表达不可由普通运行状态推导的明确停止意图；worker 在自己的锁下 ACK，fenced writer 禁止较晚的结果/完成写入。TS owner 持有 requested/acknowledged/settled/unknown/noop 决策，Python 适配信号、权威租约释放和日志 IO。CLI delegation stop --execute 与 MCP 返回同一收据，read/wait/inventory 消费持久事实；它没有授予跨 host 信号、跨 peer 权限或 Goal 结算权。
具体改动
整份差异为 17 文件、+1222/−80：停止入口、typed lifecycle、operation worker、lane/holder 只读观察、上下文/清单、派生 registry 清单、中英文档与真实进程测试。检查范围包含随分支引入的 lock/lane seam，而不是只看 stop helper；没有引入相邻 PR 的 execution-facts 变更。sidecar 有明确生产者、requester 身份与锁下更新路径，停止后 resume 拒绝重新运行；未写 stop 意图的外部 SIGTERM 保留原 recoverable 路径。
关键代码讲解
- Delegations.stop 校验现有 operation/binding，锁下保存停止意图，只向可归属的同 host worker 进程组发信号，终态返回稳定 noop。
- _acknowledge_stop 在 operation 锁内标记 stopped、记录 ACK，清理 bootstrap 并尝试释放已有硬租约；ACK 本身不是宿主清理完成的证明。
- _settle_stop 只提供 operation lock free 和 worker lane released 两项事实，再调用 decideDelegationStop；当前缺少实际 native host/后代已 drain 的事实。
- native host process cleanup 已有独立进程组与异步清理 owner，SIGTERM 后有 300ms grace 再 SIGKILL。worker/lane 释放不能替它宣告清理完成。
对主干的风险
[P1，阻断] settled 可早于实际宿主和后代终止。 在 File、SQLite 两种真实权威后端，我沿用现有 native fixture，令宿主及其同组子进程忽略 SIGTERM，其他路径仍是实际 detached worker → CLI run-once → native Node bridge → host transport。调用 stop 后约 0.26s 返回 phase=settled、operation lock free 和 lane released，但用 macOS ps 独立检查，宿主与子进程此刻都仍存活。native cleanup 稍后会杀掉它们，因此这是“过早结算”，不是声称永久 orphan；这段窗口内旧效果仍可能执行，接续工作也可能与其重叠。
原因是 _signal_worker 的提前返回 与 _settle_stop 把锁释放当成 drain；native bridge/host 位于其他进程组，拥有独立的异步清理生命周期。请复用现有 native host cleanup/readback owner，只有实际 owning host/后代清理完成才能结算；缺少可归属证明时保持 acknowledged/unknown 并允许同身份读回恢复。不要用固定 sleep、扩大超时或跨 host kill 冒充证明。补一个“宿主和同组后代忽略 TERM”的真实 File/SQLite 回归，断言返回 settled 的那一刻两者已退出，并覆盖清理中断/重复读取不产生重复接纳或完成。可放入 tests/test_local_delegation.py，重跑本段所列 pytest/TS 验证。
现有测试只在 stop 返回后再等待退出；其 /proc 检查在 macOS 读不到路径时还会把存活误当退出。请一起改为平台有效的进程检查及即时后置条件。PR 正文也应同步实现：Turn lane 是只读 liveness，operation lock 在 stop 已写后仍有短暂 kernel probe，并非全部“no lock-taking”。
语义与 CI 对齐
亲自运行 66 项 Python、18 项 TS、控制面 typecheck 均通过；独立 native drain 反例在 File/SQLite 各失败一次。这证明现有绿测试没有覆盖上述后置条件。标准 canary 14 个选择检查及 5 个直接检查中，只有 twin-budget 检查失败：原始同一 full-tree budget owner 在不可变基线 902b99698050fc21af2c3a8b0d0d3b8561d0bdd7 和本 head 都报 44/43；扫描 owner、预算和 twin 列表未被此 PR 改动，故归为既有无关失败，不作为本次 request changes 的理由。基线完整 smoke 另有旧 registry 清单行号失配，未掩盖或称其通过。
七种普通 status/Goal Chat 路径在同一夹具下，完整基线/head 观察一致；new stop 是明确请求才触发，安装、帮助发现或普通 SIGTERM 不自动激活停止。远端 CI 未查询，打包 App（无 stop 控件）、Lark、跨真实 host 及真实模型未验证。复用既有 delegation vocabulary 的范围恰当，但 settled 目前违反其停止完成语义，不能由文档描述消除。
我的整体评价
REQUEST_CHANGES。长期推进和用户体验均存在具体回归风险：用户看到结束收据时旧执行仍在 drain。规模与 CLI/MCP 到持久 ACK/读回的 R2 切片大体相称；future-facing 检查建议把 stop 协调收敛到现有 collaboration 边界、复用 native cleanup 事实，避免 Python 再建一个 host 生命周期 owner。较大的模块拆分可在有 characterization 时另作有界整理，不作为本次 LOC 门槛。当前必须先修过早结算并用原生负例证明修复，再复审精确 head；保留 rollback 对 stopped 历史的 caution，不授予自合并权限。
English verdict: REQUEST_CHANGES — exact head 5e7b7bf. A real File/SQLite native-process counterexample returns settled while the owned host and descendant remain alive. The pre-existing 44/43 twin-budget failure is unrelated and is not the blocker.
…up exit The worker and its Turn lane let go while the Host supervisor is still terminating the Host, so their release never proved that the old executor stopped. The Host transport now names the process group its supervisor owns in a record beside the operation, the drain is read back from that record without signalling anything, and the typed decision requires an exited group before a stop may settle. A group that cannot be attributed keeps an acknowledged stop open for a later same-identity read, and the TS supervisor stays the only owner that terminates a Host. Signed-off-by: song <liusongstep@gmail.com>
A real File/SQLite fixture whose host and same-group child ignore SIGTERM asserts both are gone at the instant settled returns, with a platform-valid process check in place of the /proc read that treated a live macOS process as exited. Interrupted cleanup, repeated reads, an unacknowledged holder and an unrecorded group get their own negative coverage. Signed-off-by: song <liusongstep@gmail.com>
Signed-off-by: song <liusongstep@gmail.com>
Signed-off-by: song <liusongstep@gmail.com>
|
New exact head: [P1, blocking]
|
cocolord
left a comment
cocolord
left a comment
There was a problem hiding this comment.
动机
这个 PR 要解决的是一个真实且高价值的问题：已派发的 delegated member 没有可验证的 stop，单纯杀 worker 也不能证明 Turn、原生 Host、后代进程和租约已经安全收尾。新 head 609530d2fb64e86ae71eed13ce22d4abdfd7150e 已修复我上次指出的 native Host 提前结算问题；File/SQLite 的真实进程组测试现在能证明返回 settled 的瞬间 Host 与同组 child 都已退出。
改动思路
实现复用现有 operation 单飞锁、dispatch 锁、Turn lane holder、TS collaboration decision、native Host supervisor 和 task-lease authority。stop.json 保存不可由普通运行状态推导的停止意图，host.json 保存 supervisor 产生的进程归属投影；worker 在 checkpoint 或 fenced write 处确认，decideDelegationStop 再根据 ACK、operation/lane 释放和 Host drain 选择 requested/acknowledged/settled/unknown。CLI 与 MCP 共用同一入口，read/wait/resume/inventory/context 共用持久事实。
具体改动
整份差异为 21 个文件、+1596/-81：12 个 runtime 文件增加 stop CLI/MCP、typed stopped/phase 决策、worker 信号与写 fence、holder liveness、Host spawned 回报/原子 sidecar、租约释放及各投影；7 个测试文件覆盖 TS 转移、真实进程组、File/SQLite、CLI/inventory/lane；2 个文档文件补充中英文使用与回滚语义。
关键代码讲解
Delegations.stop写入一次 stop intent，只向可归属的同机 worker 进程组发信号，并复用同一stop_id读回。_settle_stop汇总 operation lock、Turn lane 与host_process_drain；这是本轮对旧 blocker 的有效修复。decideDelegationStop是 typed phase owner，超时本身不制造 terminal receipt。runHostProcess在 Host 接收输入前报告 owned group；报告失败会取消，TS supervisor 仍是唯一 kill owner。_execute负责最终 Todo、reply 与 accepted 写入；当前剩余 blocker 都集中在 stop 与这些既有 authority 的交界。
对主干的风险
[P1，阻断] stop 与 Todo 完成/结果发布没有共同线性化边界。 _execute 在 1557 行 只做一次 pre-check，之后 _complete_delegated_todo 与 return_result 都在 dispatch fence 外提交，最后 _observe/_fenced_write 才重新看 stop。我在 File/SQLite 让 worker 停在 Todo effect 内，先通过公开 stop 写入请求，再放行 worker；两种后端都观察到 todo_completed、reply_published，随后却返回 phase=settled,status=stopped。这违反“late/other-host worker 不完成 Todo、不发布结果”的核心承诺，也可能让 successor 与已提交效果重叠。请让 stop 与两个外部 effect 通过同一 authority fence/CAS 决定谁先提交，并分别加入竞态回归。
[P1，阻断] required hard lease 释放失败仍会 terminal settle，且不会重试。 _settle_stop 的 typed 输入没有 lease release；File/SQLite 负例中 release authority 报错后仍得到 settled 和 lease_released=false，第二次 stop 原样返回且不再尝试释放。此时同 operation 已禁止 resume，但 Todo 最长仍会被 45 分钟 hard lease 阻塞。请把 required lease release 纳入 typed settlement，保留同一 stop_id 的可重试/可操作恢复路径，并覆盖失败后恢复。
[P1，阻断] Windows 上已启动 Host 的 stop 没有收敛或 fail-fast 路径。 host_process_drain 在没有 os.killpg 时恒为 unattributable；同机 signal 也依赖 killpg。通过真实 public stop 决策执行该平台分支后，即使 Host record 已是 finished，File/SQLite 都在重复调用中永久保持 acknowledged/host_process_drain_unproven。仓库已有 Windows Host tree-best-effort 支持，文档却没有把 stop 限定为 POSIX。请补 Windows supervisor/tree 完成事实和 CI，或在写入任何 stop/status 前明确 fail fast 并文档化；不能留下不可恢复的 open receipt。
验证方面，旧 blocker 的 8 个 File/SQLite native drain/cleanup-interruption 用例、另外 8 个 stop 用例、29 个 Python host/lane/CLI/inventory 用例、29 个 Node delegation/Host 用例和控制面 typecheck 均通过。远端 shard 1/3/4 的三项失败，我用同一命令在 immutable base 3ec049e138917a8cce4f84197ba196d26445b2b0 与本 head 都复现为相同的 generated-twin / prompt-upgrade 断言，因此是独立 merge-readiness hold，不是上述 request-changes 的依据。
语义与 CI 对齐
stopped 和 settled 是新的公共终态，不是提示性文案；它们会驱动 resume 拒绝、wait 返回、inventory/context 和后续协调决策。当前 typed decision 没有外部 effect 或 required lease 的事实，Windows unattributable 又没有恢复转移，因此实际语义仍宽于实现。最小修复后请重跑 File/SQLite effect race、lease failure/retry、Windows CLI/MCP stop、现有 native process suites 及完整 required CI。
我的整体评价
REQUEST_CHANGES。这个方向和范围有明确收益，stop sidecar、typed owner、只读 lane liveness 以及新的 Host group readback 都放在合理边界；上一轮 blocker 也确实修好了。但 whole-PR 的价值取决于 settled 能否成为后续继续工作的可信边界，目前 exact head 仍允许 late Todo/reply、不可重试的 lease failure，以及 Windows 永不收敛。长期推进和用户体验都因此 not_yet_proven。建议在现有 effect/lease/Host owner 上做有界修复，不再增加平行生命周期；修复后我会按新 exact head 重跑同一组正负路径。
English verdict: REQUEST_CHANGES — exact head 609530d2fb64e86ae71eed13ce22d4abdfd7150e. The prior native-Host early-settlement bug is fixed, but stop can still settle after Todo/reply effects commit, failed required lease release is terminal and never retried, and launched Windows Hosts have no converging or fail-fast stop path. Focused submitted suites pass; the three red repository shards reproduce unchanged on base and head and are unrelated.
Signed-off-by: song <liusongstep@gmail.com>
Re-review request — exact head
|
cocolord
left a comment
cocolord
left a comment
There was a problem hiding this comment.
动机
这个 PR 要解决的是一个真实且高价值的问题：已经派发的 delegated member 需要一个可验证的 stop 边界，不能只杀 worker、留下 Turn、原生 Host、Todo 或 hard lease 处于含糊状态。当前 exact head 796f688674c3a404daec89cfa6d3b5801683e292 只是把最新主干合入上次已审的 head；目标收益仍然明确，但只有当 settled 能证明后续继续或重派不会与旧执行重叠时，这个收益才成立。
改动思路
实现把不可由运行记录推导的 stop intent 放进独立 stop.json，把 Host supervisor 观察到的进程归属放进 host.json；CLI/MCP 通过 Delegations.stop 发起请求，worker 在 checkpoint 或 fenced write 处 ACK，TypeScript 的 decideDelegationStop 再组合 operation lock、Turn lane 和 Host drain 事实。正常 POSIX 路径是 requested → acknowledged → settled，read/wait/resume/inventory/context 复用同一持久事实；异常路径应当在 effect、lease 或平台清理尚未闭合时保持可恢复、不可误报 terminal。
具体改动
相对 exact base fd5f31bb3ad57448c32df6c7cddde36d07446934，整份差异为 18 个文件、+1322/-53：10 个 runtime 文件增加 stop CLI/MCP、typed stopped/phase 决策、worker 信号与写 fence、Host 进程组 sidecar、lease release 读回及 inventory/context 投影；6 个测试文件覆盖 TS 转移、真实进程组、File/SQLite、CLI 和 inventory；2 个中英文参考文档说明 stop、receipt 与回滚。先前 stacked 的 lane-holder liveness 已进入 base，本 PR 当前复用它。
关键代码讲解
Delegations.stop写入一次 operation-specific intent，只信号可归属的同机 worker，并把重复调用收敛到同一 receipt。_settle_stop读取 operation lock、Turn lane、Host drain 后调用 typed decision；目前 required lease release 并不是 decision input。decideDelegationStop是 requested/acknowledged/settled/unknown 的状态 owner，正确地拒绝用超时本身制造 settlement，但没有外部 effect 或 lease-success 事实。host_process_drain在 POSIX 上只读验证 bridge/process group；没有os.killpg时恒为unattributable。_execute的最终 effect 段 依次完成 Todo、发布 reply、再通过 fenced_observe写 accepted；stop 只在 effect 前做瞬时 pre-check。
当前 head 的第一父提交就是上次审过的 609530d…，第二父提交是 exact base；18 个 PR 文件里只有 delegation.ts、delegation_context.py、effect_runtime_handlers.ts 吸收了主干变化。下面三个 blocker 的 Python owner、Host drain 和测试文件均 byte-identical，因此不能把合主干视为修复。
对主干的风险
[P1，阻断] stop 与 Todo completion / result publication 仍没有共同线性化边界。 worker 在 1557 行 做一次 pre-check，之后两个外部 effect 都在 dispatch fence 外提交，最后写 execution record 时才再次看到 stop。exact-head 的 File/SQLite 反例都先持久化 stop，再放行 worker，仍观察到 todo_completed 和 reply_published，最终 receipt 却是 phase=settled,status=stopped。请让 stop 与两个 effect 通过同一 authority fence/CAS 决定先后，并分别提交 race regression。
[P1，阻断] required hard lease 释放失败仍被 terminal settle，且不会重试。 _settle_stop 没把 lease.released 交给 typed decision；两种后端注入一次 authority failure 后都返回 settled, lease_released=false，第二次 stop 原样返回且 release attempt 仍只有一次。此时 resume 已禁止，Todo 却可能被 hard lease 阻塞到 TTL。请把 required release 纳入 settlement obligation，并让同一 stop_id 在失败后可重试或进入明确可操作的恢复态。
[P1，阻断] Windows 上已启动 Host 的 stop 仍没有终结或 fail-fast 路径。 host_process_drain 在缺少 killpg 时恒返回 unattributable；File/SQLite 的公开 stop 路径即使读取到 phase=finished 的 Host record，重复调用仍永久停在 acknowledged/host_process_drain_unproven。通用 windows-powershell 绿灯没有覆盖这个新 stop contract。请提供 Windows supervisor/tree completion 事实并在 Windows CI 覆盖公共路径，或者在写 stop/status 前明确 fail fast 并文档化平台边界。
验证上，官方 exact-head Python focused suite 63 项、Node delegation/Host 29 项、control-plane typecheck、Ruff、diff hygiene、docs-governance 与 semantic-vocabulary smoke 均通过；独立的 6 个 File/SQLite 负例也稳定复现上述三个错误结果。绿色正向覆盖说明主体实现可运行，但没有覆盖 stop 在最后 checkpoint 之后与外部 effect/lease/platform清理交错的语义。
语义与 CI 对齐
stopped 与 settled 是机器消费的公共终态，会驱动 resume 拒绝、wait 返回、inventory/context 和后续重派，不是“guidance”。当前 typed contract 未纳入 Todo/reply commitment 和 required lease release，Windows 的 unattributable 又没有恢复转移，所以 public terminal 名称仍宽于实际保证。最小修复后请重跑 File/SQLite effect race、lease failure→retry、Windows CLI/MCP stop、现有 native Host suites 与完整 required CI。
我的整体评价
REQUEST_CHANGES。需求本身必要，stop sidecar、typed owner、只读 holder/Host facts、CLI/MCP 共用入口的方向也合理；当前 18 文件范围相对这个高风险能力并非单纯“代码太多”。但 exact head 没有修改上轮三个 blocker 的 owner，6 个独立负例全部复现，因此 before/after 的可观察收益仍达不到“安全停止并继续”的承诺，change proportionality 与 terminal authority 都是 not_yet_proven。建议只在现有 Todo/result/lease/Host authority 上做有界修复，不再增加平行生命周期；修复后按新 exact head 复审。
English verdict: REQUEST_CHANGES — exact head 796f688674c3a404daec89cfa6d3b5801683e292. This merge-only head leaves all three blockers unchanged and independently reproducible on File/SQLite: stop can settle after Todo/reply effects commit, failed required lease release is terminal and never retried, and launched Windows Hosts have no converging or fail-fast stop path. Focused Python/Node/static checks pass, but they do not cover these terminal-contract counterexamples.
…e and the platform Three blockers, all reproduced on the reviewed head before fixing. **A stop could settle after the member's effects committed.** The worker ran one pre-check and then committed Todo completion and reply publication outside any lock the stop takes, so a stop written first still reported `settled` for work that had landed. Both effects now commit inside the dispatch lock a stop also takes, so the two sides linearize: a stop written first means neither effect runs, and a stop written after leaves their acceptance intact. `_observe` gained a locked entry point because the kernel file lock is not reentrant and the widened critical section must not nest. **A failed required lease release was terminal and never retried.** `_settle_stop` did not pass the release to the typed decision, so an authority failure produced `settled` with `lease_released=false` on both backends and every later read returned the same receipt — leaving the member's Todo blocked until the lease TTL with resume already refused. `decideDelegationStop` now treats a required release as part of what `settled` promises, and the settle read retries the release under the stop's own lock. An operation that held no required lease omits the fact rather than claiming a release. **A launched Host on Windows could never settle.** `host_process_drain` returned `unattributable` whenever `killpg` was absent, so a finished Host left the stop pending forever with no converging or actionable path. That case is now `unsupported_platform`, distinct from an attribution failure, and `stop --execute` fails fast with an error naming the platform boundary instead of returning a receipt no read can settle. The public reference documents the lease obligation, the effect ordering and that boundary in both languages. Coverage: a stop written before the effects (both backends) asserts neither effect commits and the receipt still settles; a failed release asserts `acknowledged`/`required_lease_release_unproven` and that the next read retries and settles; a Host on a platform without process groups asserts the actionable failure. The TS decision pins the lease obligation on both the acknowledged and vanished-holder paths, mutation-checked by dropping it. Signed-off-by: song <liusongstep@gmail.com>
Re-review request — exact head
|
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
Exact head: 5308@01be96c65c7c1c226ab62f99f1536e9e3979072a; immutable base: fd5f31bb3ad57448c32df6c7cddde36d07446934。本次按 LoopX PR-review capability policy 12 检查完整差异和上一评审以来的修复；未查询或等待 GitHub CI。
动机
已有 delegated member 需要可验证的停止入口，直接杀 worker 不能证明 Host 后代进程、Turn lane、Todo 和 hard lease 已妥善收尾。这是 roadmap R2 的有界 CLI/MCP 生命周期增量；它不等同于整个 App/Lark/跨宿主取消或重启验收。用户需要可信的“停止完成”，随后才能处理未完成任务或启动新 operation。当前正向 stop 有效，但持久 ACK 后的 crash 会留下活跃 lease 而读回 terminal settled，持续推进和用户恢复路径仍有缺口。
改动思路
CLI 与 host-bound MCP 共用 Delegations.stop，复用固定 binding、operation single-flight、dispatch fence、原生 task-lease authority 及 TS Host supervisor。stop sidecar 保存不可由 running 状态推导的明确停止意图；worker ACK 与 lane/Host drain 分别表达不同事实，TS owner 据此决策 requested、acknowledged、settled 或 unknown。只停止原 operation，不接管 unrelated Turn，不给外部 requester 或跨 host 新权限。host.json 是 supervisor 产生的归属事实，不是用户任意写一个 pid 就能获准杀进程。
比全新团队生命周期 framework 更小的这条边界是合理的。但 lease obligation 已有 canonical operation source，不能在恢复时仅凭 sidecar 中“缺字段”推导“不需要释放”。修复应复用这一来源，让 ACK 与释放意图在持久恢复上闭合，不增加第三套 lease authority。
具体改动
完整 18 文件覆盖 CLI stop、Python collaboration service、typed observation/stop、inventory/context/effect transport、原生 Host process owner 与 bridge/transport，两份文档及 CLI、TS、真实进程回归。相对上一评审头 796f688674c3a404daec89cfa6d3b5801683e292，本头把正常完成与 reply 发布放进 stop 的 dispatch fence，增加已知 lease 释放失败的重试，并在不支持进程组证明的平台给出明确拒绝。这些改动解决了之前的具体问题，但 lease 测试只覆盖已经持久化 {required: true, released: false} 的 sidecar，遗漏了 ACK 与该字段落盘之间的 crash。
关键代码讲解
Delegations.stop（collaboration_mcp.py:932）要求 execute、验证原 binding，持久化 intent 后信号通知持锁 worker。prepared/running/turn_returned 可停止；accepted/rejected 是无副作用 noop；stopped operation 不再 resume 或复用旧 id。_acknowledge_stop（:1039）在 operation 与 dispatch 边界下把状态改为 stopped，并先保存 ACK。之后才清 bootstrap、释放 lease、再次保存 lease 结果。它正确区分 worker/requester ACK，但这两次持久化之间存在恢复窗口。_settle_stop（:1102）读 operation lock、lane holder、Host drain，并对 sidecar 已知的 required lease 重试释放。第 1129–1139 行只看stop.lease，没有从row.task_lease.required恢复 obligation；缺值时不向 TS 传lease_released: false。decideDelegationStop（delegation.ts:367）持有 typed phase 决策，缺少lease_released被当作可结算。Python 调用端未正确区分“不需要 lease”与“释放事实尚未写下”，因此 TS 收到完整 ACK/drain 事实后返回 settled。- Host process transport 与 TS spawn supervisor 记录独立 Host 进程组，停止会等到原 Host 与同组后代退出；inventory 过滤 stop/host sidecar，context 投影 stopped。SIGTERM 没有 stop intent 时仍走原可恢复行为，而非偷偷转换成 cancel。
对主干的风险
[P1] ACK 后 crash 会把仍有活跃 hard lease 的 stop 永久标为 settled
位置：collaboration_mcp.py:1129–1139，上游写窗口是第 1070–1085 行。
独立验证在隔离的 synthetic Goal 中，通过真实 native authority acquire 取得 required lease，给原 operation 保存其实际身份与 version；只在 _release_delegation_lease 的入口注入进程退出，以模拟 ACK 已持久化、释放尚未发生。没有 mock lease 的最终状态或 settlement 结果。此时 operation 已 stopped、ACK 存在，但 stop.lease 仍是 null。恢复真实 release 实现，再从 public stop(..., execute=True) 读回，得到 settled + null lease；独立 native inspect_task_lease 却仍报告 active。真实 File 与 SQLite provider 都复现，2/2 未满足独立的终态后置条件。
_settle_stop 把 null 变成空对象，跳过 required-lease retry，再把 settled 固定成 terminal；后续读回不再修复。用户已收到“停止完成”，但原 Todo 仍被旧 lease 阻塞至 TTL。最低修复：ACK 落盘时保存不可丢失的 required lease obligation/identity，恢复时同时依据 operation 的 canonical required lease 判断，释放或独立确认释放后才允许 settled。不要把缺失 receipt 等同于无 obligation。回归须在 ACK、release 与 receipt 各持久化边界注入 process loss，重新构建 service，再验证 native lease readback；覆盖 File/SQLite，不用“mock release 返回 true”作为后置条件。
本次 62 项 Python（含 CLI/MCP、真实 worker/Host/child/lane）及 29 项相关 TS 测试通过；typecheck 与改动路径 ruff 通过。独立 source premerge 的 5 direct、5 catalog、8 risk、1 public/private boundary 均通过；但缺失 crash case 不会被这些绿灯证明正确。相同普通未请求 stop 的任务在 immutable base/head、File/SQLite 上各 2/2 通过：一次 Host、相同 accepted/canonical completion/readback、无 stop sidecar，默认路径没有被这个新增入口污染。所有测试使用该源码环境；没有 paid model call，也没有修改活跃项目状态。
语义与 CI 对齐
settled 的当前 obligation 包含实际 required lease 已释放，不只是停止了本地进程；本 PR 扩展既有 delegation observation vocabulary 和 local stop contract，不能把 machine-enforced drain/lease 条件称作 advisory。状态采用 typed phase/decision owner，不是文字或 substring 分类，通用错误仍 domain-neutral。违规是 Python producer 的未知事实被编码成“没有 required lease”，不是 enum 名称本身。复审命令须覆盖 tests/test_local_delegation.py、tests/test_delegation_cli.py、tests/control_plane/test_host_process.py、相关 TS tests，加上上述真实 authority crash-recovery 对照。此旧头的 inventory 不支持新的 changed-diff advisory 参数；支持的完整 inventory 报告及 full-tree semantic drift 均通过。CI 不参与本次判定。
我的整体评价
English verdict: REQUEST_CHANGES — exact head 01be96c65c7c1c226ab62f99f1536e9e3979072a; process loss after durable ACK produces terminal settled while a native required lease remains active, reproduced on real File/SQLite authority.
本地 stop 的目标、边界与复用方式合理，不需要重做整套架构或因为代码量拒绝。previous-review-to-head 已实质改善 effect linearization、已记录的 lease failure retry 和平台诊断；但完整 PR 的 long_horizon 与 user_experience 仍因终态失真判为 regression。面向下一次改动的有界 refine 应在现有 ACK/settle owner 收敛 lease obligation 的唯一来源，并用持久化边界测试覆盖，而不是添加互相同步的布尔旗标。真实 stop intent 与历史 operation 兼容义务应保留；未验证 App/Lark、跨宿主、PostgreSQL 或 Windows 原生执行，不据模拟平台测试声称这些验收已完成。修复上述 P1 后再复审；当前不建议合并，也没有自合并权限。
`_acknowledge_stop` persists the ACK before it releases the hard lease, so a process loss between those two writes leaves the sidecar with no `lease` field at all. `_settle_stop` read the obligation from that field, took the absence as "no required lease", and settled: the caller was told the member is safely stopped while the native authority still reported its lease active, with resume already refused and the Todo blocked until the TTL. Reproduced on both File and SQLite by acquiring a real required lease and losing the process at the release entry. The obligation now comes from the operation record — the canonical `row.task_lease.required`, which the crash cannot lose. Absent sidecar state means "the release result has not been written yet", not "nothing was owed", so the release is attempted and only an actual release lets the decision settle. That is one source of truth rather than a second boolean kept in sync with it; the released fact is still persisted beside the stop so a later read does not release twice. Coverage pins both ends of the window on both authorities: - a crash after the ACK with the lease still owed, and a release that succeeds on the next read, settles and reports `lease_released: true`; - the same window with the release still failing stays `acknowledged` with `required_lease_release_unproven` instead of reporting `settled`. Mutation-checked: deriving the obligation from the sidecar again fails all four. Signed-off-by: song <liusongstep@gmail.com>
Signed-off-by: song <liusongstep@gmail.com>
Re-review request — exact head
|
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
复审结论：REQUEST_CHANGES，当前 head 8468544d0a8bd8526fa336d655701a64e74918b8。上一轮 ACK/lease 崩溃窗口的修复已验证；整份 PR 仍有一个 P1：恢复已通过验证、尚未结算的 Turn 时，Todo completion 没有进入 stop 的同一把 dispatch 锁，因此 stop 可以先落盘，Todo 随后被完成，最终仍返回 settled/stopped。
动机
现有 coordinator pause 只暂停协调者，已经启动的 delegated member 仍可能继续执行。这个 PR 让操作者经 CLI/MCP 明确停止一个 member，并区分“已请求”“已确认”“已完成清理”，保留证据，由协调者选择新的 operation 继续任务。这是有用的有界增量：单纯发 SIGTERM 无法证明原生 Host、worker、lane 和必需 lease 都已释放；不需要扩展成整个团队、跨主机或 App 的停止协议。
从普通任务看，最短流程应是：定位 operation → 显式 stop → 读取可信回执 → 检查未完成 Todo → 用新 operation 继续。恢复路径把已停止的 Todo 标成完成，会破坏最后两步；这不是仅影响诊断文本的问题。
改动思路
继续复用既有 delegation 生命周期、原生 Turn/lease authority 和 TS Host supervisor。停止意图保存于独立 .stop.json，Host attribution 保存于 .host.json；这些分别承载无法从执行状态推导的用户意图和实际进程身份。decideDelegationStop 在现有 TS owner 内根据 ACK、锁、lane、Host drain 和 lease release 事实推导 phase，Python 负责文件/进程 IO，没有另建一套 Goal completion authority。
stopped 是该 operation 的终态，旧 id 禁止 resume；停止不等于任务完成。已 accepted/rejected 的记录保持原结论，普通 SIGTERM 也不自动成为有授权的 stop。这个设计比强行终止进程或把超时当作成功更完整，新增 CLI、持久意图和 readback 的成本与问题相称。
具体改动
本轮按当前 merge base 67930ab6af78491f10ca3de4ff74ef7a39954a51 审查全部 18 个文件（+1666/-76），而非只看上一轮修复：
- CLI/MCP：新增
delegation stop --execute/stop_delegation，接入现有绑定和 operation 身份；read/wait返回 stop 状态。没有新增停止整个 Goal 或授予 peer 独立权限的入口。 - Worker/transport：stop 与 dispatch 共用 fence，启动记录传入原生 Host transport；复用 TS supervisor 的进程组清理，Python 只观察是否已 drained。外地主机、无法证明 drain、超时和 lease release 未证明都不能冒充 settled。
- TS 状态与投影：delegation transition/stop decision、effect handler、inventory、context 和 subagent context 一起识别
stopped；sidecar 不混入 operation inventory，旧 operation 的 resume 被拒绝。 - 文档与测试：两份文档解释 coordinator pause、单 member stop、清理回执和新 operation 恢复；6 个测试文件覆盖 CLI、inventory、TS transition 和 Host/worker。App/Lark UI 没有改变，本次可用入口明确是 CLI/MCP，不能据此宣称已交付 App 停止按钮。
关键路径：stop（collaboration_mcp.py:990）写停止意图；_acknowledge_stop（1097）记录 ACK 并尝试释放 lease；_settle_stop（1160）重读权威事实；TS decideDelegationStop（delegation.ts:367）决定回执 phase。普通完成路径（1663–1688）把 Todo completion、结果发布和 accepted transition 放入同一把 dispatch 锁。
相对上一轮 01be96c…，最新修复从 operation 的 task_lease.required 推导释放义务，避免 ACK 已持久化、lease 结果尚未写入时把缺字段误认为无需释放。以隔离的真实 File/SQLite authority 和 hard lease 分别注入“释放前失去进程”“释放后但结果落盘前失去进程”，重新实例化服务、公开 stop 及独立 native lease inspect，4 个用例全部通过。这个旧 blocker 已关闭；仓库新增测试本身使用 mocked release，所以本轮补了实际 authority 验证。期间合入的 workspace preflight/main 变更以当前 base 为准，不算本 PR 的新增交付。
对主干的风险
[P1] 恢复 Turn 的 completion 仍可越过已生效的 stop。 位置：恢复路径的检查与 completion。
当 turn_result 尚未 committed、但 _validated_turn_journal 已确认真实 Host 输出和 task validation 时，1626 行只检查 stop，1627 行在 dispatch 锁外调用 _complete_delegated_todo。若 stop 在这个检查之后、completion 之前先落盘，native completion 仍会执行；1629 行才看到 stop 并进入 ACK/settlement。后面的 1663 行锁无法保护已经执行过的这个 effect。
独立反例在 File 和 SQLite 两种后端均复现：使用实际 Host/task validation 输出构造持久化的可恢复 journal 边界，在恢复分支调用 completion 之前经公开 stop 写入意图，然后让原 completion 经 native CLI/authority 执行。最终公开回执是 phase=settled, status=stopped，独立读取 canonical Todo 却是 done=true。测试的独立 oracle 是“stop 先落盘则 Todo 仍打开”，两个用例均失败。这里注入了恢复 journal 和确定的 interleaving，并抑制了同步测试进程的 worker signal；没有声称用真实 OS crash 或真实信号调度复现这一竞态，也没有 mock completion 或 canonical done 后置条件。
最小修复：让恢复路径的 stop 检查与 Todo completion 也通过现有 dispatch fence 线性化，并检查后续结算/结果发布对同一 operation 的一致性。补两个后端的回归：stop 先赢时不完成 Todo、不发布成功结果，返回停止回执；completion 先赢时保留真实已提交事实，不将其改报为未完成。不要只在 completion 前后再加一次无锁检查。
验证：73 项相关 Python 测试、29 项 TS 测试、control-plane typecheck、changed Python ruff、完整 premerge（含语义词汇、维护性、输出预算及 public/private boundary）通过。相同 synthetic harness 在 immutable base/head 的 File/SQLite 普通无 stop 路径均得到 accepted、一次 Host 调用、一次结果返回、Todo done，且没有 stop projection/sidecar；当前 head 的两个 operation scope/恢复用例也通过：同 Goal 的未覆盖 sibling 保持 prepared，旧 id resume 被拒，新 id 能实际执行到 accepted。上述通过项没有覆盖 P1 的恢复 interleaving，不能抵消它。按当前 review policy 未查询或等待远端 CI。
我的整体评价
停止能力的需求、归属和整批范围成立，上一轮 lease 修复也成立。当前阻塞是同一 completion effect 有两个路径，其中恢复路径遗漏 fence，导致终态与 canonical Todo 不一致。建议在现有 owner 内统一这个 effect 的线性化入口；这也是本轮 future-facing 检查认为最有价值的有界简化，避免下一次恢复规则变更再次漏掉一条路径，不需要新框架或语言迁移。
保留的验证边界：隔离源 checkout 和 synthetic native Host，未运行付费模型、跨主机停止、PostgreSQL 或 Windows 进程组；本 PR 明示的单主机边界不因这些未测维度被扩大。本次请求修复 P1 并重新验证上述先后顺序，再对新 exact head 复审。PR 保持打开，合并留给维护者。
English verdict: REQUEST_CHANGES - head 8468544. The ACK/lease crash-window fix passes real File/SQLite lease recovery checks. However, validated-Turn recovery completes the canonical Todo outside the dispatch fence: a persisted stop can win first, yet the native completion commits and the final receipt says settled/stopped. Reproduced with an injected recoverable journal and deterministic interleaving on both backends, without mocking completion. 73 Python tests, 29 TS tests, typecheck, ruff, baseline/head no-stop parity, scoped continuation and premerge pass. Fence this recovery effect and add both ordering cases before approval.
This branch has not been deployed
3 participants
"## Why\n\nRoadmap R2 requires that stop, cancel and restart retain work and fence old executors. Delegated members could not be stopped at all. The worker held its operation lock for the whole run and rewrote the execution record from memory, so a stop written into that record could neither reach it nor survive it, and a killed run left its Turn and hard lease unaccounted for. Pausing the coordinator or disabling LoopX mode does not stop admitted members either.\n\n## What changed\n\n- Typed state and decision (TS).
stoppedis a terminal observation reachable from prepared, running and turn_returned. The newcollaboration.delegation.stopdecision settles a stop only with an acknowledgement from the worker, a free operation lock, a released Turn lane and an exited native Host process group. Free locks without an acknowledgement areunknown, never a fabricated settlement, and elapsed time alone never produces a receipt.\n- Stop receipt beside the record.executions/<h>/<op>.stop.jsonis written only under the existing.dispatchlock, never into the execution record. Phases are requested, acknowledged, settled, with terminalsunknownandnoop. It records the requester, the worker (pid, process group, host), the acknowledgement, the lease release and the settlement facts.\n-Delegations.stop. A terminal record returns an identicalnoopreceipt on every call. With no running worker the requester marks the record stopped, releases the hard lease and settles. A same-host worker gets SIGTERM on its process group, and SIGKILL only if it still holds the operation after the grace period. Another host's worker is never signalled and finds the request itself.\n- Settled means the host is gone too. The Turn names the process group its Host supervisor owns inexecutions/<h>/<op>.host.json, written before the request is sent and updated when the supervisor reports the spawned group; the supervisor reports before the Host gets its input, so no Host runs unattributed.settledadditionally requires that group to have exited, read back read-only while the TS supervisor stays the only owner that terminates a Host. A group that cannot be attributed, or one still terminating, keeps the stopacknowledgedso a later same-identity read can settle it.\n- Worker and fencing. A SIGTERM handler and checkpoints before running, before each run-once and before completing the Todo raise on a stop. Every record write re-reads the stop under.dispatchand refuses a stop this process did not acknowledge, so a late or other-host worker records no Turn result and completes no Todo. The in-progress Turn journal is kept for inspection.\n- No lock-taking lane probes. Settlement reads the lane's last holder record through the lock-freeturn_lane_liveness: released, dead or absent frees the lane; a live same-host holder frees it only outside the recorded worker's process group; another host's or an unattributable holder keeps the stop open, so a stop never refuses a legitimate Turn of the same member. The operation lock has no holder-record reader yet, so once the stop is written settlement probes that kernel lock for an instant;resume, its only single-flight acquirer, refuses a stopped operation before it touches the lock. A regression test races a real lane acquisition against settlement.\n- Surfaces.loopx delegation stop --operation-id ID --executeand thestop_delegationMCP tool return the same receipt.resumerefuses stopped work,waitreturns on stopped,readshows the stop phase, and inventory and context projections count stopped receipts.\n- Docs.local-delegation.mdandgoal-chat-continuation.mddescribe stopping a member and reading its receipt, in English and Chinese.\n\n## Stacking and coordination\n\n- This branch includes #5306's commitfeat(turn-driver): read Turn lane liveness without taking the laneunchanged, to reuselock_holder_host_label()and the lock-free lane read. It drops out once #5306 merges; review fromfeat(delegation): add stopped observation.\n- #5304 editscollaboration_mcp.pyanddelegation.tsat adjacent insertion points. The second to merge rebases with a keep-both resolution; #5304 treats any terminal non-accepted status, includingstopped, as no wake.\n\n## Checks run on this head\n\n| Check | Result |\n|---|---|\n|pytest tests/test_local_delegation.py tests/test_delegation_cli.py tests/test_delegation_inventory.py tests/test_collaboration_mcp.py tests/test_turn_lane_fence.pyplus module budget, import boundary and registry census tests | 79 passed |\n|node --test tests/control_plane_ts/delegation.test.ts| 18 passed, 0 failed |\n|npm run typecheck:control-plane| ok |\n|examples/docs-governance-smoke.py,examples/semantic-vocabulary-drift-smoke.py| ok |\n|loopx canary premerge --from-git-diff| passed: tier=standard, changed_files=18, surfaces=control_plane/docs_project_content/public_boundary/python; selected=16, failures=0 |\n\nNot run: stopping a member on another real host (covered by fixtures only), the packaged App and Lark. The App has no stop control yet; CLI and MCP are the entry points.\n\n## Rollback\n\nRevert the PR. Stop receipts stay as files beside their records and the older code ignores them. Caution: the olderresumetreats every status other thanacceptedandrejectedas unfinished, so after a revert aresumeon astoppedoperation would restart it. Checkloopx delegation operationsfor stopped operations before reverting, and do not resume them afterwards.\n\n## Bounded future-facing refactor\n\nApplied: stop settlement reuses the file-lock owner's holder records instead of a second host helper and a lock-taking probe. Deferred: an App stop control and cross-host stop acknowledgement, which need R6 host identity.\n\nThis is a control-plane change; it is left for maintainer review and merge.\n\n\ud83e\udd16 Generated with Claude Code\n"